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DETAILED ACTION 

This action is in response to the amendment filed on 06/05/06. 

Claim Rejections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

3. Claims 1, 4 - 6, 8 t 12, 13, 15, 17, 19, 23, and 38 - 43 are rejected under 35 
U.S.C. 103(a) as being unpatentable over the combination of Grimson et al ("Using 
adaptive tracking to classify and monitor activities in a site", Proceedings 1998 IEEE 
Conference on Computer Vision and Pattern recognition, Santa Barbara, CA, 1998 
June 23 - 25; pages 22 - 29) and Stein ("Tracking from multiple view points: Self- 
calibration of space and time", Computer Vision and Pattern Recognition, 1999, IEEE 
Computer Society Conference on, Volume 1, 23-25 June 1999 Page(s): 521 - 527). 

In regards to claim 1, A method for use in monitoring a search area, the method 
comprising: positioning a plurality of imaging devices to provide image pixel data 
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covering an entire search area defined by an outer perimeter edge (Section 3: Grimson 
discloses creating a "composite video" from a plurality of cameras. The composite 
image would be contained within an outer perimeter edge. That edge might not be a 
rectangle, but it would outline the entire composite image.); providing a plurality of 
frames of image pixel data, wherein the plurality of frames of image pixel data include at 
least one frame of image pixel data representative of a corresponding field of view for 
each of a plurality of imaging devices, wherein each field of view of each imaging device 
comprises a field of view portion which overlaps with at least one other field of view of 
another imaging device (Section 3); [selecting and physically marking the defined 
search area with a plurality of non-movable landmark points of commonality in field of 
view portions which overlap] (Section 3: Grimson teaches of finding a homography 
transform matrix by tracking moving objects, in this case cars and people.); combining 
the plurality of frames of image pixel data into a single image representative of the 
entire search area by computing at least one [homography transformation matrix using 
image pixel data corresponding to the plurality of physically marked landmark points] of 
commonality in field of view portions which overlap indicative of a coordinate 
relationship between image pixel data for fields of view of at least one pair of the 
imaging devices that comprise field of view portions which overlap with each other, 
wherein the field of view portion which overlaps is greater than [25 percent] of the field 
of view of the imaging device (Section 3); and tracking a moving object within the single 
image resulting in a moving object path for the moving object (Section 3). 
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Grimson teaches of tracking multiple objects in a "composite image" created 
using homography transform matrices. These matrices are created by tracking moving 
objects in the video streams. Grimson does not teach that a homography matrix should 
be created from the non-moving, i.e., static, objects in the video streams. Also, 
although it seems clear that Grimson envisions a system that has overlaps in excess of 
25% Grimson does not state this. Although the figures would be used as proof of this 
concept unfortunately the examiner was unable to obtain clear pictures that would 
accurately prove the overlap. Thus Stein, which will be used to show how Grimson 
could be improved by also taking into account static objects, will be used to show that 
an overlap of 25% is desirable. 

Stein discloses an improvement to the system disclosed by Grimson, by first 
finding (Sections 1.1 and 3.3) a course homography matrix by tracking moving objects 
just as Grimson discloses. Stein notes that "the initial alignment does not perfectly align 
the ground plane". Stein then refines the alignment by finding a homography transform 
matrix in using the static features of the video streams. Finally figures 1(a) and 1(c) 
clearly show two image streams to be merged, the image streams overlap each other 
by at least 25%. 

Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to use the refined homography transform approach disclosed by 
Stein in the method taught by Grimson. The combination would allow for a more 
accurate alignment of the video streams. Thus more accurate information would be 
obtained from the composite video stream. By overlapping large areas of the video 
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streams more there is a better chance of matching the frames accurately as more area 
will be provided, thus allowing for more matches to be found. 

In regards to claim 4, claim 4 is rejected for the same reasons as claim 1 . The 
argument analogous to that presented above for claim 1 is applicable to claim 4. Please 
see figures 1 (a) and 1 (c) for further evidence. 

In regards to claim 5, claim 5 is rejected for the same reasons as claim 1. The 
argument analogous to that presented above for claim 1 is applicable to claim 5. 

In regards to claim 6, The method of claim 1, wherein combining the plurality of 
frames of image pixel data into a single image representative of at least a portion of the 
search area further comprises using the homography transformation matrix to fuse the 
plurality of frames of image pixel data into a single image having a global coordinate 
system (Section 3). 

In regards to claim 8, claim 8 is rejected for the same reasons as claim 1 . The 
argument analogous to that presented above for claim 1 is applicable to claim 8. 

In regards to claim 12, The method of claim 11, wherein the method further 
comprises determining whether the moving object path is normal or abnormal (p. 26, 
first paragraph under section 5.1 and first two paragraphs under section 5.2). 

In regards to claims 13, 15, 17, 19, and 23, claims 13, 15, 17, 19, and 23 are 
rejected for the same reasons as claims 1, 4, 6, 8, and 12. The argument analogous to 
that presented above for claims 1, 4, 6, 8, and 12 is applicable to claims 13, 15, 17, 19, 
and 23. 
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In regards to claims 38 and 39, claims 38 and 39 are rejected for the same 
reasons as claims 1 and 4. The argument analogous to that presented above for claims 
1 and 4 is applicable to claims 38 and 39. 

In regards to claim 40, claim 40 is rejected for the same reasons as claim 1 . The 
argument analogous to that presented above for claim 1 is applicable to claim 40. 

In regards to claim 41, claim 41 is rejected for the same reasons as claim 1. The 
argument analogous to that presented above for claim 1 is applicable to claim 41 . 

In regards to claim 42, claim 42 is rejected for the same reasons as claim 1 . The 
argument analogous to that presented above for claim 1 is applicable to claim 42. 

In regards to claim 43, claim 43 is rejected for the same reasons as claim 12. 
The argument analogous to that presented above for claim 12 is applicable to claim 43. 
4. Claims 9, 10, 20, 21, 24, 26 - 30, 31, 33 - 37 are rejected under 35 U.S.C. 
103(a) as being unpatentable over the combination of Grimson and Stein. 

In regards to claims 9 and 10, both Grimson and Stein disclose multiple cameras, 
which are positioned in such a fashion as to get a complete image of an aerial scene. It 
is clear that both references place cameras at more than one installation site to obtain 
the overall images. Please note the examiner is construing the term installation site as a 
physical structure which cameras can be mounted on. So for example, a light pole, a 
building, etc. Although it seems likely that Grimson and Stein both position multiple 
cameras from a single installation site (a building). However neither reference clearly 
states this. 



Application/Control Number: 10/034,696 Page 7 

Art Unit: 2624 

It would have been obvious to one having ordinary skill in the art at the time of 
the invention to place multiple cameras at one installation site in the combination taught 
by Grimson and Stein. Grimson and Stein are both attempting to obtain a mosaic 
image of a large area obtained from a high vantage point. As both references are for 
tracking traffic patterns, which would comprise of street scenes in cities an obvious 
choice to obtain this vantage point would be buildings. Doing so would allow for images 
from the same height and therefore easier to combine into a single mosaic image. 

In regards to claims 20 and 21, claims 20 and 21 are rejected for the same 
reasons as claims 9 and 10. The argument analogous to that presented above for 
claims 9 and 10 is applicable to claims 20 and 21. 

In regards to claim 24, claim 24 is rejected for the same reasons as claim 9. The 
argument analogous to that presented above for claim 9 is applicable to claim 24. 

In regards to claim 26, claim 26 is rejected for the same reasons as claim 4. The 
argument analogous to that presented above for claim 4 is applicable to claim 26. 

In regards to claim 27, claim 27 is rejected for the same reasons as claim 10. 
The argument analogous to that presented above for claim 10 is applicable to claim 27. 

In regards to claims 28 and 29, claims 28 and 29 are rejected for the same 
reasons as claim 1. The argument analogous to that presented above for claims 28 and 
29 is applicable to claim 1 . 

In regards to claim 30, claim 30 is rejected for the same reasons as claim 12. 
The argument analogous to that presented above for claim 12 is applicable to claim 30. 
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In regards to claims 31 and 33 - 37, claims 31 and 33 - 37 are rejected for the 
same reasons as claims 24 and 26 - 30. The argument analogous to that presented 
above for claims 24 and 26 - 30 is applicable to claims 31 and 33 - 37. 

Response to Arguments 

5. Applicant's arguments filed 06/05/06 have been fully considered but they are not 
persuasive. 

6. The examine would first like to thank the applicant for pointing out the examiner's 
typo with regards to claim 1 where the examiner stated "greater than 85%" instead of 
"greater than 25%". This mistake has been corrected, and the examiner would like to 
apologize for any confusion the typo caused. 

7. The applicant first agues "the references cited do not show such selection and 
physical marking". The examiner would like to point out that the applicant is entitled to 
the broadest possible interpretation of the language and thus physical marking can be 
read as any operation used to identify non-moving points of commonality between 
images. Stein discloses that static features are used to form a homography matrix. 
Thus Stein discloses physically marking non-moving points of commonality. 

8. The applicant's primary argument centers on the Irani document supplied by the 
applicant. Although at first blush the applicant's arguments appear to be valid with 
regards to Irani upon closer inspection it is clear that Irani does in fact relay on static 
features and not motion information. Irani is addressing a problem called Ego-motion 
although it would probably be better understood if it were called camera shaking. 
Essentially when a user is holding a video camera, the users hands will shake slightly 
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leading to a jittery picture. Irani is trying to use software to correct for this jittering by 
warping the pictures together so the image is smooth. Throughout the article when 
motion is being talked about it is the motion of the camera itself and not the motion of 
the scene. First paragraph of the introduction "The motion observed in an image 
sequence can be caused by camera motion (ego-motion) and by motion of objects 
moving in the scene. In this paper we address the case of the camera moving in a static 
scene. " 

Stein obviously is not dealing with the same situation (a single handheld 
camera). However, after the initial alignment Stein's warped images are rather close to 
what one would expect from ego-motion between frames of video. So Stein's 
descriptions of the operation are correct. "Given the rough alignment we can use robust 
estimation techniques on the static features to determine a more accurate alignment of 
the ground plane". And, "the general idea is to search for a homography matrix A2 that 
minimizes the 'sum square difference' between image 1 and the warped image 2". So 
to summarize, Irani shows that Stein is using static features to find the difference (using 
homography) between image 1 and warped image 2, this difference is used to refine the 
alignment. 

Conclusion 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
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TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Christopher L. Lavin whose telephone number is 571- 

272- 7392. The examiner can normally be reached on M - F (8:30 - 5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh M. Mehta can be reached on (571) 272-7453. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




